Skip to content

Read remote files: client.file(path) with byte ranges, streaming and digests (#146) - #177

Merged
bertysentry merged 2 commits into
mainfrom
feature/146-remote-file-read
Sep 24, 2026
Merged

bertysentry merged 2 commits into
mainfrom
feature/146-remote-file-read

Conversation

@bertysentry

Copy link
Copy Markdown
Contributor

Closes #146 — first issue of the remote file access family (#146 → #145 listing, #147 download, #148 CLI).

What

byte[] content = client.file("C:\\Windows\\Temp\\collect.bin").readBytes();
String tail = client.file("D:\\logs\\huge.log").offset(-8192).readText(StandardCharsets.UTF_8);
try (InputStream in = client.file(path).openStream()) { ... }   // bounded memory, no cap
String sha = client.file(path).digest("SHA256");
  • WinRMClient.file(path) → RemoteFile request: offset(long) (negative = from the end, resolved on the host from the open stream), length(long), maxBytes(long) (64 MiB default cap for the in-memory terminals, also sent to the host), timeout(Duration).
  • Package-private RemoteFiles primitive: a PowerShell script (-EncodedCommand, path embedded base64-encoded: no quoting) opens the file with a ReadWrite share mode (logs held open by a service can be read), seeks, and writes each block as one base64 line to the raw stdout stream. The client decodes lines as they arrive; the exit code, checked at the end of the output, becomes an explicit WinRMClientException (not found, directory, access denied, sharing violation, Constrained Language Mode, PowerShell missing). A missing file fails in openStream() itself; a failure midway is never a silently short read.
  • Docs: new files.md page (menu, index, cross-link from file-transfers.md), README one-liner.

Performance

Investigated in depth: the WinRM service reads a command's stdout pipe itself, at most 32 KiB per read, ~60 reads/s per stream — not CPU (host 97–99 % idle; client ingests 24–31 MB/s). Blocks are therefore 49,149 bytes, so each 65,533-byte line fills exactly two reads. Measured with a 20 MiB file, SHA-256 verified:

Host before (Console.Out lines) 57 KiB raw writes 49,149-byte blocks (this PR)
Windows Server 2008 R2 / PS 2.0 101 s 17.2–18.5 s 13.7–15.1 s
Windows Server 2019 / PS 5.1 — 18.6–18.8 s 15.2–15.5 s
Windows Server 2022 / PS 5.1 101 s 17.5–18.4 s 14.3–14.9 s

Follow-ups opened from these measurements: #175 (raw bytes instead of base64, ~+33 %) and #176 (parallel ranged downloads, ~3× with 4 connections).

Tests

  • RemoteFilesScriptTest runs the generated scripts in the local powershell.exe (Windows only, skipped elsewhere): every byte value across blocks, non-ASCII name, ranges, negative offsets, a file held open by a writer, a locked file → sharing violation, exit codes, digest.
  • RemoteFileTest (FakeWsmanServer): base64 lines split across Receive chunks, a UTF-8 character split across blocks, BOM kept, size cap sent and enforced, failures, unexpected output, wall-clock timeout.
  • WinRMLiveTest: byte-exact read of a non-ASCII-named file, tail range, digest; 1 GiB tail served by a seek; a file held open for writing by another session. Passed on anaxagore (2008 R2) and tc-win2022.
  • mvn clean verify site green: no checkstyle/PMD/SpotBugs findings.

Deviations from the issue

  • readText(Charset) / openReader(Charset) take the charset as a parameter (no charset() setter, no default to forget).
  • digest() hashes with .NET on the host, reusing the same script and exit-code mapping, rather than ShellFileCopy's certutil probe (which goes through cmd.exe quoting).

🤖 Generated with Claude Code

…digests (#146)

Foundation of the remote file access family (#146, then #145, #147, #148):
client.file(path) returns a RemoteFile request with readBytes(),
readText(Charset), openStream(), openReader(Charset) and digest(String),
plus offset(long) (negative = from the end), length(long), maxBytes(long)
(64 MiB default cap) and timeout(Duration).

The package-private RemoteFiles primitive runs a small PowerShell script
(-EncodedCommand, path embedded base64-encoded, no quoting) that opens the
file with a ReadWrite share mode, seeks, and writes each block as one
base64 line to the raw stdout stream; the client decodes the lines as they
arrive and maps the script's exit code to explicit failures (not found,
directory, access denied, sharing violation, Constrained Language Mode,
PowerShell missing).

Blocks are 49,149 bytes so each 65,533-byte line fills exactly two of the
WinRM service's 32 KiB pipe reads (at most ~60 per second per stream):
20 MiB now reads in 14-15 s on Windows Server 2008 R2, 2019 and 2022,
instead of 101 s with Console.Out-written 76 KB lines.

Tests: the scripts run in the local powershell.exe on Windows
(RemoteFilesScriptTest), client-side decoding against FakeWsmanServer
(RemoteFileTest), and live tests; docs in the new files.md page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-24T12:22:46.478668Z c642860 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc9693dbe5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread README.md Outdated
Comment thread src/main/java/org/metricshub/winrm/RemoteFile.java
Comment thread src/main/java/org/metricshub/winrm/RemoteFiles.java
Comment thread src/main/java/org/metricshub/winrm/RemoteFile.java
…ADME

- readBytes(): a read that fills exactly the requested length now drains the
  stream to its end, so the script's exit code is still checked and a late
  failure is not reported as a successful range read.
- A path too long for the reader script to fit the remote command line
  (about 1,450 ASCII characters) is refused with a WinRMClientException
  before anything is sent, instead of silently falling back to uploading the
  script: a read never writes on the host. Documented in files.md.
- readText()/openReader() Javadoc: nothing is stripped, a byte order mark is
  handled as the given charset's decoder handles it (UTF-16 consumes it).
- README: revert the quick-start addition (AGENTS.md keeps README terse; the
  feature is documented in files.md).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@bertysentry

Copy link
Copy Markdown
Contributor Author

@codex please review again

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: c64286047e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remote file access (1/4): foundation + read remote file content — client.file(path), RemoteFiles primitive, whole file, byte ranges, text, streaming

1 participant